Imaging apparatus, information processing apparatus, image processing system, and control method

ABSTRACT

The present invention is directed to enabling the capturing of an image to facilitate the comparison between affected parts. An imaging apparatus includes an image capturing unit, and a control unit configured to perform control to, in a case where posture information regarding an object obtained when an image of an affected part of the object has been captured in the past is acquired and the image capturing unit captures an image of the affected part of the object, notify a user of the posture information regarding the object and inclination information regarding the imaging apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Patent ApplicationNo. PCT/JP2020/008448, filed Feb. 28, 2020, which claims the benefit ofJapanese Patent Applications No. 2019-045041, filed Mar. 12, 2019, andNo. 2020-023400, filed Feb. 14, 2020, all of which are herebyincorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an imaging apparatus, an informationprocessing apparatus, an image processing system, and a control method.

Background Art

In a state where a person or an animal lies down, a part of the body incontact with a contact surface may be compressed by body weight, therebydeveloping a pressure ulcer, i.e., a bedsore. To a patient who hasdeveloped a pressure ulcer, it is necessary to provide pressure ulcercare such as body pressure dispersion care and skin care. Then, it isnecessary to periodically evaluate and manage the pressure ulcer.

On page 23 of Shorinsha, Guidebook for Pressure Ulcers (second edition),in compliance with Guidelines for the Prevention and Management ofPressure Ulcers (fourth edition) (edited by the Japanese Society ofPressure Ulcers, ISBN13 978-4796523608), DESIGN-R (registeredtrademark), which is a pressure ulcer status determination scaledeveloped by the Scientific Education Committee of the Japanese Societyof Pressure Ulcers, is discussed as a tool for pressure ulcerevaluation. DESIGN-R is a tool for evaluating a healing process of awound such as a pressure ulcer. The name of the scale is an acronym ofDepth, Exudate, Size, Inflammation/Infection, Granulation, and Necrotictissue, which are observation items.

DESIGN-R includes two types, namely DESIGN-R for classification ofseverity for daily simple evaluation, and DESIGN-R for progressevaluation indicating steps in the healing process in detail. DESIGN-Rfor classification of severity classifies six evaluation items into twolevels, namely mild and severe levels. The mild level is representedusing lowercase alphabetic characters, and the severe level isrepresented using uppercase alphabetic characters.

A pressure ulcer is evaluated using the classification of severity in aninitial treatment, whereby it is possible to grasp a general state ofthe pressure ulcer. It identifies which item is problematic, and thus, atreatment strategy can be easily determined.

Meanwhile, as DESIGN-R for progress evaluation, DESIGN-R capable ofcomparing severity between patients in addition to providing theprogress evaluation is also defined. R represents rating (evaluation orgrading). Each item is weighted differently, and a total point (0 to 66points) of the six items except for Depth indicate the severity of thepressure ulcer. After the treatment is started, progress of thetreatment can be evaluated in detail and objectively. Thus, it ispossible not only to evaluate the progress for an individual, but alsoto compare the severity between patients.

In Size evaluation of DESIGN-R, Size is classified into seven levels,where Size is a numerical value obtained by measuring, in centimeters,the major axis and the minor axis (the maximum diameter orthogonal tothe major axis) of the extent of skin injury and by multiplying themajor axis and the minor axis. The seven levels are: s0 indicating noskin injury, s3 indicating Size is less than 4, s6 indicating Size is 4or more and less than 16, s8 indicating Size is 16 or more and less than36, s9 indicating Size is 36 or more and less than 64, s12 indicatingSize is 64 or more and less than 100, and s15 indicating Size is 100 ormore.

In scoring of DESIGN-R, as discussed in Guidebook for Pressure Ulcers,it is recommended that scoring be performed once a week or two weeks toevaluate progress in healing of the pressure ulcer and selectappropriate care. Thus, it is necessary to periodically evaluate andmanage a medical condition of the pressure ulcer. Accuracy is requiredin the evaluation to check a change in the medical condition of thepressure ulcer.

CITATION LIST Non-Patent Literature

NPL 1: Japanese Society of Pressure Ulcers. (2015), Guidebook forPressure Ulcers (second edition): in compliance with Guidelines for thePrevention and Management of Pressure Ulcers (fourth edition).Shorinsha.

SUMMARY OF THE INVENTION

However, in a case where a pressure ulcer is captured, the shape, thearea, and the shape of a pocket of the pressure ulcer change dependingon posture of the patient. Thus, every time the pressure ulcer iscaptured, visibility of the pressure ulcer may change. Thus, it has beendifficult to accurately compare the progress of the pressure ulcer bycomparing images obtained by capturing the pressure ulcer. This is notlimited to a pressure ulcer, and the same applies to a case where a burnor a laceration is captured.

The present invention is directed to enabling the capturing of an imageto facilitate the comparison between affected parts.

According to an aspect of the present invention, an imaging apparatusincludes an image capturing unit, and a control unit configured toperform control to, in a case where posture information regarding anobject obtained when an image of an affected part of the object has beencaptured in the past is acquired and the image capturing unit capturesan image of the affected part of the object, notify a user of theposture information regarding the object and inclination informationregarding the imaging apparatus.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a functional configuration of an imageprocessing system.

FIG. 2 is a diagram illustrating an object.

FIG. 3 is a diagram illustrating a hardware configuration of an imagingapparatus.

FIG. 4 is a diagram illustrating a hardware configuration of aninformation processing apparatus.

FIG. 5, which includes FIGS. 5A and 5B, is a flowchart illustratingprocessing of the image processing system.

FIG. 6 is a diagram illustrating a calculation method for calculating anarea of an affected region.

FIG. 7A is a diagram illustrating a method for superimposing informationon image data of an affected part.

FIG. 7B is a diagram illustrating a method for superimposing informationon image data of an affected part.

FIG. 8A is a diagram illustrating a method for superimposing informationon image data of an affected part.

FIG. 8B is a diagram illustrating a method for superimposing informationon image data of an affected part.

FIG. 8C is a diagram illustrating a method for superimposing informationon image data of an affected part.

FIG. 9A is a diagram illustrating a data configuration of objectinformation.

FIG. 9B is a diagram illustrating a data configuration of objectinformation.

FIG. 10A is a diagram illustrating image data including postureinformation.

FIG. 10B is a diagram illustrating image data including postureinformation.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described belowwith reference to the drawings.

First Exemplary Embodiment

FIG. 1 is a diagram illustrating an example of a functionalconfiguration of an image processing system 1.

The image processing system 1 includes an imaging apparatus 200 that isa handheld portable device, and an information processing apparatus 300.

FIG. 2 is a diagram illustrating an example of an object 101 that is apatient whose affected part is evaluated by the image processing system1. In the present exemplary embodiment, as an example of a clinicalcondition of an affected part 102 developed in the buttocks of theobject 101, a pressure ulcer developed in the buttocks is described.

A barcode tag 103 is attached to the object 101. The barcode tag 103includes patient identification (ID) as identification information foridentifying the object 101. Thus, the image processing system 1 canmanage the identification information regarding the object 101 and imagedata obtained by capturing an image of the affected part 102 inassociation with each other. The identification information is notlimited to the barcode tag 103, and may be a two-dimensional code suchas a QR code (registered trademark) or a numerical value, or may be dataor an ID number attached to an ID card such as a patient registrationcard.

In the image processing system 1, the imaging apparatus 200 capturesimages of the affected part 102 of the object 101 and the barcode tag103 as the identification information and transmits the images to theinformation processing apparatus 300. The information processingapparatus 300 transmits posture information regarding the object 101obtained when an image of the affected part 102 of the same object 101has been captured in the past, as posture information associated withthe received identification information to the imaging apparatus 200.The imaging apparatus 200 performs display based on the received postureinformation, whereby a user can grasp the posture of the object 101taken when the image of the affected part 102 of the same object 101 hasbeen captured in the past. The posture information may only need toinclude at least information that allows identification of the postureof the object 101 to be any one of a prone posture, a recumbent posture(a right lateral recumbent posture or a left lateral recumbent posture),and a sitting posture. While the present exemplary embodiment isdescribed using an example in which the affected part 102 is a pressureulcer, the affected part 102 is not limited to a pressure ulcer and maybe a burn or a laceration.

FIG. 3 is a diagram illustrating an example of a hardware configurationof the imaging apparatus 200.

As the imaging apparatus 200, a general single-lens reflex camera, acompact digital camera, or a smartphone or a tablet terminal including acamera having an autofocus function can be used.

An image capturing unit 211 includes a lens group 212, a shutter 213,and an image sensor 214. By changing positions of a plurality of lensesincluded in the lens group 212, a focus position and a zoommagnification can be changed. The lens group 212 also includes adiaphragm for adjusting an amount of exposure.

The image sensor 214 is composed of a charge accumulation type solidstate image sensor such as a charge-coupled device (CCD) sensor or acomplementary metal-oxide-semiconductor (CMOS) sensor, which converts anoptical image into electric data. Reflected light from the object 101having passed through the lens group 212 and the shutter 213 forms animage on the image sensor 214. The image sensor 214 generates anelectric signal corresponding to an object image and outputs image databased on the generated electric signal.

The shutter 213 performs operation of opening and closing a blademember, thereby exposing the image sensor 214 and blocking light fromreaching the image sensor 214. Thus, the shutter 213 controls anexposure time of the image sensor 214. The shutter 213 may be anelectronic shutter that controls the exposure time by driving the imagesensor 214. In a case where the electronic shutter is implemented usinga CMOS sensor, a reset scan for setting an amount of accumulated chargeof each pixel or an amount of accumulated charge of pixels in eachregion (e.g., each line) including a plurality of pixels to zero isperformed. Then, for each pixel or each region subjected to the resetscan, after a predetermined time elapses, scanning for reading a signalcorresponding to the amount of accumulated charge is performed.

A zoom control circuit 215 controls a motor for driving a zoom lensincluded in the lens group 212, thereby controlling an opticalmagnification of the lens group 212.

A distance measurement system 216 calculates distance informationregarding a distance to the object 101. The distance measurement system216 may generate the distance information based on output of anautofocus (AF) control circuit 218. In a case where there is a pluralityof areas to be AF targets in a screen, the distance measurement system216 may cause the AF control circuit 218 to repeatedly perform an AFprocess on each area, thereby generating the distance information foreach area. The distance measurement system 216 may use a time-of-flight(ToF) sensor. The TOF sensor is a sensor that measures a distance to aphysical body based on a time difference (or a phase difference) betweena transmission timing of an irradiation wave and a reception timing of areflected wave of the irradiation wave reflected by the physical body.Further, the distance measurement system 216 may use a positionsensitive device (PSD) method using a PSD as a light-receiving element.

An image processing circuit 217 performs predetermined image processingon the image data output from the image sensor 214. The image processingcircuit 217 performs various types of image processing such as whitebalance adjustment, gamma correction, color interpolation, demosaicing,and filtering on image data output from the image capturing unit 211 orimage data stored in an internal memory 221. The image processingcircuit 217 also performs a compression process based on a standard suchas the Joint Photographic Experts Group (JPEG) standard on the imagedata subjected to the image processing.

Based on the distance information obtained by the distance measurementsystem 216, the AF control circuit 218 determines the position of afocus lens included in the lens group 212 and controls a motor fordriving the focus lens. The AF control circuit 218 may perform TV-AF, orcontrast AF, for extracting and integrating a high-frequency componentof the image data and determining a position of the focus lens at whichthe integral value is the greatest. The focus control method is notlimited to the contrast AF and may be phase difference AF or another AFmethod. Further, the AF control circuit 218 may detect an amount offocus adjustment or a position of the focus lens and, based on theposition of the focus lens, acquire distance information regarding thedistance to the object 101.

A communication device 219 is a communication interface forcommunicating with an external device such as the information processingapparatus 300 using a wireless network. Specific examples of the networkinclude a network based on the Wi-Fi (registered trademark) standard.Communication using Wi-Fi may be implemented via a router.Alternatively, the communication device 219 may be implemented by awired communication interface based on the Universal Serial Bus (USB)standard or the local area network (LAN) standard.

A system control circuit 220 includes a central processing unit (CPU).The system control circuit 220 executes a program stored in the internalmemory 221, thereby controlling the entire imaging apparatus 200. Thesystem control circuit 220 also controls the image capturing unit 211,the zoom control circuit 215, the distance measurement system 216, theimage processing circuit 217, and the AF control circuit 218. The systemcontrol circuit 220 is not limited to a configuration including a CPUand may include a field-programmable gate array (FPGA) or anapplication-specific integrated circuit (ASIC).

As the internal memory 221, for example, a rewritable memory such as aflash memory or a synchronous dynamic random-access memory (SDRAM) canbe used. The internal memory 221 temporarily stores various pieces ofsetting information required for the operation of the imaging apparatus200, such as information regarding a focus position when an image iscaptured, the image data captured by the image capturing unit 211, andthe image data subjected to the image processing by the image processingcircuit 217. The internal memory 221 may temporarily store the imagedata and analysis data on information regarding the size of the object101 that are received by the communication device 219 communicating withthe information processing apparatus 300.

An external memory 222 is a non-volatile recording medium attachable tothe imaging apparatus 200 or built into the imaging apparatus 200. Asthe external memory 222, for example, a Secure Digital (SD) card or aCompactFlash (CF) card can be used. The external memory 222 records theimage data subjected to the image processing by the image processingcircuit 217 and the image data and the analysis data received by thecommunication device 219 communicating with the information processingapparatus 300. When reproduction is performed, the image data recordedin the external memory 222 is read and can be output to outside theimaging apparatus 200.

As a display device 223, for example, a thin-film transistor (TFT)liquid crystal display, an organic electroluminescent (EL) display, oran electronic viewfinder (EVF) can be used. The display device 223displays the image data temporarily stored in the internal memory 221 orthe image data recorded in the external memory 222, or displays asetting screen for the imaging apparatus 200.

An operation unit 224 includes a button, a switch, a key, and a modedial that are provided in the imaging apparatus 200 or a touch panelthat also serves as the display device 223. The system control circuit220 is notified of a command such as a mode setting or an imagecapturing instruction from the user via the operation unit 224.

An inclination detection device 225 detects an inclination of theimaging apparatus 200. In the present exemplary embodiment, theinclination of the imaging apparatus 200 refers to an angle based on thehorizontal direction. As the inclination detection device 225, forexample, a gyro sensor or an acceleration sensor can be used.

A common bus 226 is a signal line for transmitting and receiving asignal between the components of the imaging apparatus 200.

FIG. 4 is a diagram illustrating an example of a hardware configurationof the information processing apparatus 300.

The information processing apparatus 300 includes a CPU 310, a storagedevice 312, a communication device 313, an output device 314, and anauxiliary calculation device 317.

The CPU 310 includes a calculation device 311. The CPU 310 executes aprogram stored in the storage device 312, thereby controlling theentirety information processing apparatus 300 and also implementing thefunctional configuration of the information processing apparatus 300illustrated in FIG. 4.

The storage device 312 includes a main storage device 315 (a read-onlymemory (ROM) or a random-access memory (RAM)) and an auxiliary storagedevice 316 (a magnetic disk device or a solid-state drive (SSD)).

The communication device 313 is a wireless communication module forcommunicating with an external device such as the imaging apparatus 200using a wireless network.

The output device 314 outputs data processed by the calculation device311 or data stored in the storage device 312 to a display, a printer, oran external network connected to the information processing apparatus300.

The auxiliary calculation device 317 is an auxiliary calculationintegrated circuit (IC) that operates under control of the CPU 310. Asthe auxiliary calculation device 317, a graphics processing unit (GPU)can be used. The GPU is originally an image processing processor, butcan also be used as a processor that performs processing for signallearning because the GPU includes a plurality of product-sum calculatorsand excels at matrix calculations. Thus, the GPU is generally used inprocessing for performing deep learning. As the auxiliary calculationdevice 317, for example, Jetson TX2 Module manufactured by NvidiaCorporation can be used. Alternatively, as the auxiliary calculationdevice 317, an FPGA or an ASIC may be used. The auxiliary calculationdevice 317 performs an extraction process for extracting an affectedregion from image data.

The information processing apparatus 300 may include the single CPU 310or a plurality of CPUs 310 and include the single storage device 312 ora plurality of storage devices 312. In other words, at least one or moreCPUs and at least one or more storage devices are connected together,and if the at least one or more CPUs execute a program stored in the atleast one or more storage devices, the information processing apparatus300 executes functions described below. The information processingapparatus 300 may include not only the CPU but also an FPGA or an ASIC.

FIG. 5, which includes FIGS. 5A and 5B, is a flowchart illustrating anexample of the processing of the image processing system 1.

In FIGS. 5A and 5B, steps S501 to S519 are the processing performed bythe imaging apparatus 200, and steps S521 to S550 are the processingperformed by the information processing apparatus 300. The flowchart inFIGS. 5A and 5B is started by the imaging apparatus 200 and theinformation processing apparatus 300 connecting to a network based onthe Wi-Fi standard, which is a wireless LAN standard.

In step S521, the CPU 310 of the information processing apparatus 300performs, via the communication device 313, a search process in searchof the imaging apparatus 200 to connect to.

In step S501, the system control circuit 220 of the imaging apparatus200 performs, via the communication device 219, a response process inresponse to the search process performed by the information processingapparatus 300. As a technique for searching for a device via a network,Universal Plug and Play (UPnP) is used. In PnP, an individual apparatusis identified by a universally unique identifier (UUID).

In step S502, using the display device 223, the system control circuit220 prompts the user to capture an image of entire body posture of anobject from which a posture of the object when an image of an affectedpart is captured can be understood, and an image of a barcode tag foridentifying the object. In response to an image capturing instructionfrom the user, the image capturing unit 211 captures the images of theposture of the object and the barcode tag of the object.

In this step, before the image of the affected part of the object iscaptured, the object is asked to take, for example, a prone posture, arecumbent posture, or a sitting posture. Then, the user captures theimage of the entire body posture of the object from which the posture ofthe object when the image of the affected part is captured can beunderstood. At this time, based on inclination information output fromthe inclination detection device 225, the system control circuit 220generates inclination information regarding the imaging apparatus 200when the posture is captured.

Next, processing for live view in steps S503 to S511 is described.

In step S503, the AF control circuit 218 performs an AF process forcontrolling the driving of the lens group 212 so that the object comesinto focus.

In this step, since it is assumed that the user holds the imagingapparatus 200 so that the affected part is at the center of the screen,the AF control circuit 218 performs the AF process in an area at thecenter of the screen. Based on the amount of focus adjustment or theamount of movement of the focus lens, the AF control circuit 218 outputsthe distance information regarding the distance to the object.

In step S504, using the display device 223, the system control circuit220 prompts the user to capture an image of the affected part of theobject. In response to an image capturing instruction from the user, theimage capturing unit 211 captures an image of the object.

In step S505, the image processing circuit 217 acquires data of thecaptured image and performs a development process and a compressionprocess on the image data, thereby generating image data based on theJPEG standard, for example. The image processing circuit 217 performs aresizing process on the image data subjected to the compression process,thereby reducing the size of the image data.

The imaging apparatus 200 will transmit the image data subjected to theresizing process using wireless communication in step S508 describedbelow. The larger the size of the image data to be transmitted is, thelonger time it takes to perform the wireless communication. Thus, instep S505, based on an acceptable communication time, the system controlcircuit 220 determines the size of the image data to be subjected to theresizing process and gives an instruction to the image processingcircuit 217.

In step S532 described below, the information processing apparatus 300will extract an affected region from the image data subjected to theresizing process. The size of the image data influences a time taken toextract the affected region and accuracy of the extraction. Thus, instep S505, based on the time taken to extract the affected region andthe accuracy of the extraction, the system control circuit 220determines the size of the image data to be subjected to the resizingprocess.

The resizing process in step S505 is processing performed during thelive view. Thus, if a processing time is long, a frame rate of live viewimages becomes low. Thus, in step S505, it is desirable that the systemcontrol circuit 220 perform the resizing process in which the image datais resized to a smaller size than or the same size as that in a resizingprocess in step S514 described below, which is not the processingperformed during the live view.

In the present exemplary embodiment, the image data is resized to a sizeof approximately 1.1 megabytes in the case of 720 pixels×540 pixels in8-bit red, green, and blue (RGB) colors. However, the size of the imagedata to be subjected to the resizing process is not limited to theabove.

In step S506, the system control circuit 220 generates distanceinformation regarding the distance to the object. Specifically, based onthe distance information output from the AF control circuit 218, thesystem control circuit 220 generates the distance information regardingthe distance from the imaging apparatus 200 to the object. If the AFcontrol circuit 218 performs the AF process on each of a plurality ofareas in the screen in step S503, the system control circuit 220 maygenerate the distance information with respect to each of the pluralityof areas. As for a method for generating the distance information, thedistance information regarding the distance to the object calculated bythe distance measurement system 216 may be used.

In step S507, based on the inclination information output from theinclination detection device 225, the system control circuit 220generates inclination information regarding the imaging apparatus 200 inthe live view.

In this step, since it is assumed that the user holds the imagingapparatus 200 so that an image capturing range thereof includes theaffected part, the system control circuit 220 generates inclinationinformation regarding the imaging apparatus 200 when the user holds theimaging apparatus 200 and points the imaging apparatus 200 at theaffected part.

In step S508, the system control circuit 220 transmits various pieces ofinformation to the information processing apparatus 300 via thecommunication device 219. Specifically, the system control circuit 220transmits the image data of the affected part subjected to the resizingprocess in step S505, the distance information regarding the distance tothe object that is generated in step S506, and the inclinationinformation regarding the imaging apparatus 200 in the live view that isgenerated in step S507. The system control circuit 220 also transmitsimage data of the posture captured in step S502, the inclinationinformation regarding the imaging apparatus 200 when the posture iscaptured, and image data of the barcode tag to the informationprocessing apparatus 300. Patient ID included in the image data of thebarcode tag is not information that changes. Thus, regarding the samepatient, the system control circuit 220 transmits the image data of thebarcode tag only the first time. Regarding the same patient, the systemcontrol circuit 220 also transmits the image data of the posture and theinclination information regarding the imaging apparatus 200 when theposture is captured only once the first time.

Next, the processing performed by the information processing apparatus300 will be described.

In step S531, the CPU 310 of the information processing apparatus 300receives, via the communication device 313, the image data of theaffected part, the distance information regarding the distance to theobject, and the inclination information regarding the imaging apparatus200 in the live view that are transmitted from the imaging apparatus200. Regarding the same patient, the CPU 310 receives the image data ofthe posture, the inclination information regarding the imaging apparatus200 when the posture is captured, and the image data of the barcode tagonly the first time.

In step S532, using the auxiliary calculation device 317, the CPU 310extracts an affected region from the received image data of the affectedpart (segments the affected region and another region). As a techniquefor the region segmentation, semantic region segmentation using deeplearning is performed. More specifically, a learning computer is trainedin advance on a model of a neural network using a plurality of images ofaffected regions of actual pressure ulcers as supervised data, therebygenerating a trained model. The auxiliary calculation device 317acquires the trained model from the computer and estimates a pressureulcer area from the image data based on the trained model. As an exampleof the model of the neural network, a model of a fully convolutionalnetwork (FCN), which is a segmentation model using deep learning, can beapplied. An inference of deep learning is processed by the GPU that isincluded in the auxiliary calculation device 317 and that excels atparallel execution of product-sum calculations. However, the inferenceof deep learning may be executed by an FPGA or an ASIC. Alternatively,the region segmentation may be implemented using another model of deeplearning. The segmentation technique is not limited to the deeplearning, and for example, graph cuts, region growing, edge detection,or a divide-and-conquer method may be used. Further, a model of theneural network may be trained within the auxiliary calculation device317 using the images of affected regions of pressure ulcers as thesupervised data.

In step S533, the calculation device 311 of the CPU 310 calculates thearea of the affected region as information regarding the size of theextracted affected region. The calculation device 311 converts the sizeof the extracted affected region in the image data based on informationregarding the angle of view or the pixel size of the image data and thedistance information generated by the system control circuit 220,thereby calculating the area of the affected region.

FIG. 6 is a diagram illustrating a calculation method for calculatingthe area of the affected region.

In a case where the imaging apparatus 200 is a general camera, theimaging apparatus 200 can be treated as a pinhole model as illustratedin FIG. 6. Incident light 601 passes through a lens principal point of alens 212 a and is received by an imaging surface of the image sensor214. The distance from the imaging surface to the lens principal pointis a focal length F602. In a case where the lens group 212 isapproximated to the single lens 212 a that is not thick, two principalpoints, namely a front principal point and a rear principal point, ofthe lens 212 a can be regarded as coinciding with each other. The focusposition of the lens 212 a is adjusted so that an image is formed on aflat surface of the image sensor 214, whereby the imaging apparatus 200can focus on an object 604. The focal length F602, which is the distancefrom the imaging surface to the lens principal point, is changed,thereby changing an angle of view θ603. This changes the zoommagnification. At this time, based on a relationship between the angleof view θ603 of the imaging apparatus 200 and an object distance D605,an object width W606 on a focal plane is geometrically determined. Theobject width W606 is calculated using a trigonometric function. Morespecifically, the object width W606 is determined based on therelationship between the angle of view θ603 that changes depending onthe focal length F602, and the object distance D605. A value of theobject width W606 is divided by the number of pixels on a correspondingline of the image sensor 214, thereby acquiring the length on the focalplane corresponding to one pixel on the image.

The calculation device 311 calculates the area of the affected region asthe product of the number of pixels in the region obtained from a resultof the region segmentation in step S532, and the area of one pixelobtained from the length on the focal plane corresponding to one pixelon the image. A formula for obtaining the object width W606 or thelength on the focal plane corresponding to one pixel on the image may berecursively obtained by acquiring data while changing the objectdistance D605 and capturing an object of which the object width W606 isknown.

In a case where the object distance D605 is a single distance, thecalculation device 311 can correctly obtain the area of the affectedregion on the premise that the object 604 is a flat surface and the flatsurface is perpendicular to the optical axis. If, however, the distanceinformation is generated with respect to each of the plurality of areasin step S506, the calculation device 311 may detect the inclination ofor a change in the object in the depth direction, and based on thedetected inclination or change, calculate the area of the affectedregion.

In step S534, the image processing circuit 217 generates image dataobtained by superimposing information indicating a result of extractionof the affected region and information regarding the size of theaffected region on the image data from which the affected region is tobe extracted.

FIGS. 7A and 7B are diagrams illustrating a method for superimposing theinformation indicating the result of extraction of the affected regionand the information regarding the size of the affected region on theimage data.

An image 701 illustrated in FIG. 7A is an example of display of imagedata before the superimposition process is performed, and includes theobject 101 and the affected part 102. An image 702 illustrated in FIG.7B is an example of display of image data after the superimpositionprocess is performed.

At the upper left corner of the image 702 illustrated in FIG. 7B, alabel 711 is superimposed in which a character string 712 indicating thearea of the affected region is displayed in white characters on a blackbackground. In the image 702, the information regarding the size of theaffected region is the character string 712 and is the area of theaffected region calculated by the calculation device 311. The backgroundcolor and the color of the character string of the label 711 are notlimited to black and white as long as the colors are easy to see.Further, an amount of transmission may be set and alpha blending(a-blending) may be performed, so that the user can check the image in aportion where the label 711 is superimposed.

On the image 702, an indicator 713 indicating an estimated area of theaffected region extracted in step S532 is superimposed. The indicator713 indicating the estimated area and the image data from which theimage 701 is generated are subjected to a-blending, whereby the user cancheck whether the estimated area from which the area of the affectedregion is calculated is appropriate. It is desirable that the color ofthe indicator 713 indicating the estimated area be different from thecolor of the object 101. It is also desirable that a range oftransmittance of the a-blending be a range where the estimated area andthe original affected part 102 can be distinguished from each other. Ifthe indicator 713 indicating the estimated area of the affected regionis displayed in a superimposed manner, the user can check whether theestimated area is appropriate even if the label 711 is not displayed.Thus, step S533 may be omitted.

In step S535, the CPU 310 reads the patient ID from the image data ofthe barcode tag. In step S536, the CPU 310 checks the read patient IDagainst the patient ID of the object registered in advance in thestorage device 312, thereby acquiring information regarding the name ofthe object.

In step S537, the CPU 310 stores the image data of the affected part inassociation with the patient ID and the information regarding the nameof the object in the storage device 312. Until the CPU 310 receivesimage data of a barcode tag that is subsequently captured, the CPU 310processes the image data of the affected part received in step S531 asdata regarding the same patient ID and the same information regardingthe name of the object.

The CPU 310 also determines whether object information corresponding tothe target patient ID is stored in the storage device 312. If the objectinformation corresponding to the target patient ID is not stored, theCPU 310 generates object information corresponding to the patient ID andthe information regarding the name of the object. On the other hand, ifthe object information corresponding to the target patient ID is alreadystored in the storage device 312, the processing proceeds to step S538.

FIG. 9A is a diagram illustrating an example of a data configuration ofobject information 900. The object information 900 is managed withrespect to each patient ID.

The object information 900 includes a patient ID field 901, aname-of-object field 902, posture information 903, and affected partinformation 908.

The patient ID field 901 stores the patient ID. The name-of-object field902 stores the name of the object.

The posture information 903 includes a posture icon field 904, animage-data-of-posture field 905, a first inclination information field906, and a second inclination information field 907. The posture iconfield 904 stores a posture icon schematically illustrating the postureof the object when the image of the affected part is captured oridentification information regarding the posture icon. The posture iconcorresponds to an example of a display item.

FIG. 9B is a diagram illustrating examples of the posture icon.

A posture icon 921 is an icon representing a prone posture. A postureicon 922 is an icon representing a right lateral recumbent posture withthe right side down. A posture icon 923 is an icon representing a leftlateral recumbent posture with the left side down. A posture icon 924 isan icon representing a sitting posture.

The image-data-of-posture field 905 stores the image data of the postureobtained by capturing the posture of the object in step S502 or addressinformation regarding an address where the image data of the posture isstored.

The first inclination information field 906 stores the inclinationinformation regarding the imaging apparatus 200 when the posture iscaptured in step S502. The second inclination information field 907stores inclination information regarding the imaging apparatus 200 inimage capturing for recording in which the live view is ended and theimage of the affected part is captured as a record. The secondinclination information field 907 stores the inclination informationregarding the imaging apparatus 200 when the image capturing forrecording is performed for the first time or the last time with thetarget patient ID, or an average value of pieces of inclinationinformation regarding the imaging apparatus 200 when the image capturingfor recording is performed multiple times. The inclination informationin the second inclination information field 907 is stored or updatedbased on inclination information regarding the imaging apparatus 200 inthe image capturing for recording that is stored in an inclinationinformation field 912. When performing the image capturing forrecording, the user references the inclination information stored in thesecond inclination information field 907 and thereby can use theinclination information to cause the imaging apparatus 200 to face thesurface of the affected part.

The posture information 903 may store information that allowsidentification of the posture of the object, such as characterinformation representing the posture of the object in characters“prone”, “sitting”, “right lateral recumbent”, or “left lateralrecumbent”.

The affected part information 908 includes animage-capturing-date-and-time field 909, an image-data-of-affected-partfield 910, an evaluation information field 911, and the inclinationinformation field 912. The image-capturing-date-and-time field 909stores a date and time of the image capturing for recording performed instep S513 described below. The image-data-of-affected-part field 910stores image data of the affected part obtained by the image capturingfor recording or address information regarding an address where theimage data of the affected part is stored. The evaluation informationfield 911 stores information indicating a result of evaluation of theaffected region. The inclination information field 912 storesinclination information regarding the imaging apparatus 200 in the imagecapturing for recording.

If the object information 900 corresponding to the target patient ID isnot stored in step S537, the CPU 310 adds information to the postureicon field 904, the image-data-of-posture field 905, and the firstinclination information field 906 in the posture information 903 of thegenerated object information 900, and stores the resulting informationin the storage device 312. Specifically, to add information to theposture icon field 904, first, based on the image data of the posturereceived by the auxiliary calculation device 317 in step S531, the CPU310 determines to which of the posture icons 921 to 924 illustrated inFIG. 9B the posture of the object corresponds. Next, the CPU 310 storesthe posture icon or the identification information regarding the postureicon in the posture icon field 904. The CPU 310 also stores the imagedata of the posture received in step S531 in the image-data-of-posturefield 905. Further, the CPU 310 stores the inclination informationregarding the imaging apparatus 200 when the posture is captured that isreceived in step S531 in the first inclination information field 906.

On the other hand, if the object information 900 corresponding to thetarget patient ID is stored in step S537, this means that an image ofthe affected part has been captured in the past and pieces ofinformation are already stored in the posture information 903 and theaffected part information 908 of the object information 900. Thus, theprocessing proceeds to step S538.

In step S538, the CPU 310 of the information processing apparatus 300transmits the information indicating the result of extraction of theaffected region and the information regarding the size of the affectedregion to the imaging apparatus 200 via the communication device 313. Inthe present exemplary embodiment, the CPU 310 transmits the image dataobtained by superimposing the information indicating the result ofextraction of the affected region and the information regarding the sizeof the affected region on the image data of the affected part that isgenerated in step S534 to the imaging apparatus 200.

To notify the user of the posture of the object taken when the affectedpart has been captured in the past, the CPU 310 transmits the postureinformation 903 of the object information 900 to the imaging apparatus200 via the communication device 313. Specifically, the CPU 310transmits the posture icon, the image data of the posture, theinclination information regarding the imaging apparatus 200 when theposture is captured, and the inclination information regarding theimaging apparatus 200 in the image capturing for recording. In a casewhere the CPU 310 transmits, multiple times during the live view, theimage data obtained by superimposing the information indicating theresult of extraction of the affected region and the informationregarding the size of the affected region on the image data of theaffected part, the CPU 310 transmits the posture information 903 onlythe first time. Alternatively, the CPU 310 may transmit the inclinationinformation regarding the imaging apparatus 200 in the live view that isreceived in step S531. If the object information 900 corresponding tothe target patient ID is not stored in step S537 because the imagecapturing for recording has not been performed in the past, informationis not stored in the second inclination information field 907. Thus, theinclination information regarding the imaging apparatus 200 in the imagecapturing for recording is not transmitted.

Then, the processing performed by the imaging apparatus 200 will bedescribed.

In step S509, the system control circuit 220 of the imaging apparatus200 receives, via the communication device 219, the image data obtainedby superimposing the information indicating the result of extraction ofthe affected region and the information regarding the size of theaffected region on the image data of the affected part that istransmitted from the information processing apparatus 300. The systemcontrol circuit 220 also receives, via the communication device 219, theposture icon, the image data of the posture, the inclination informationregarding the imaging apparatus 200 when the posture is captured, andthe inclination information regarding the imaging apparatus 200 in theimage capturing for recording that are transmitted from the informationprocessing apparatus 300.

In step S510, the system control circuit 220 displays, on the displaydevice 223, the image data obtained by superimposing the informationindicating the result of extraction of the affected region and theinformation regarding the size of the affected region on the image dataof the affected part. The information indicating the result ofextraction of the affected part is thus displayed in a superimposedmanner on the image data in the live view, whereby the user can confirmwhether the estimated area and the area of the affected region areappropriate, and then proceed to the image capturing for recording.

The system control circuit 220 also displays, on the display device 223,at least any posture information among the posture icon, the image dataof the posture, and the inclination information regarding the imagingapparatus 200 when the posture is captured that are received. The useris thus notified of the posture information regarding the objectobtained when the image of the affected part has been captured in thepast. The system control circuit 220 may display the inclinationinformation regarding the imaging apparatus 200 in the image capturingfor recording and the inclination information regarding the imagingapparatus 200 in the live view.

FIGS. 10A and 10B are diagrams illustrating examples of image dataincluding the posture information. Portions similar to those in FIGS. 7Aand 7B are indicated by the same reference numerals, and the descriptionof the similar portions is appropriately omitted.

An image 1001 illustrated in FIG. 10A is an example of display of imagedata obtained by superimposing a posture icon 1002 on the image 702illustrated in FIG. 7B.

The system control circuit 220 displays, on the display device 223, theimage 1001 obtained by superimposing the posture icon 1002, which isbased on the posture icon or the identification information regardingthe posture icon that is received in step S509, on the image 702illustrated in FIG. 7B.

In FIG. 7B, the posture icon 1002 functions as a button on which theuser can perform a touch operation through the touch panel that alsoserves as the display device 223. In response to the touch operation bythe user on the posture icon 1002, the system control circuit 220transitions the screen and displays an image 1003 illustrated in FIG.10B.

The image 1003 illustrated in FIG. 10B is an example of display of theimage data of the posture. At the upper left corner of the image 1003, alabel 1006 including inclination information 1004 and a character string1005 is displayed in white characters on a black background.

The system control circuit 220 displays, on the display device 223, theimage 1003 obtained by superimposing the label 1006 on the image data ofthe posture received in step S509. Based on the inclination informationregarding the imaging apparatus 200 when the posture is captured that isreceived in step S509, the system control circuit 220 displays theinclination information 1004. In a case where the posture informationreceived in step S509 includes the character information representingthe posture, the system control circuit 220 displays the characterstring 1005 of the label 1006 based on the character informationregarding the posture.

As described above, before the image of the affected part is capturedfor recording, the user is notified of the posture information regardingthe object obtained when the image of the affected part of the sameobject has been captured in the past, whereby the user can grasp theposture taken when the image of the affected part of the object has beencaptured in the past. Thus, the user can ask the object to take the sameposture as the posture taken when the image of the affected part hasbeen captured in the past, and thereby can appropriately capture theimage of the affected part of the object.

Specifically, the posture icon 1002 schematically illustrating theposture of the object is displayed, whereby the user can immediatelygrasp the posture of the object taken when the image of the affectedpart of the object has been captured in the past. The image 1003obtained by capturing the posture of the object is also displayed,whereby the user can accurately grasp the posture of the object takenwhen the image of the affected part of the object has been captured inthe past. Further, the inclination information 1004 regarding theimaging apparatus 200 is displayed, whereby the user can grasp theinclination of the imaging apparatus 200 when the posture is captured.However, an image in which the posture information is to be displayed isnot limited to the images illustrated in FIGS. 10A and 10B, and may beany image as long as the user can grasp the posture of the object.

The system control circuit 220 may display the inclination informationregarding the imaging apparatus 200 in the image capturing for recordingthat is received in step S509. The user references the displayedinclination information and thereby can capture the image of theaffected part at an inclination similar to that when the image of theaffected part has been captured in the past. Thus, the user can causethe imaging apparatus 200 to face the surface of the affected part.

At this time, the system control circuit 220 may display the inclinationinformation regarding the imaging apparatus 200 in the live view that isgenerated in step S507 or received in step S509. In this case, the usercan reference the inclination of the imaging apparatus 200 at thecurrent moment and thus can match the current inclination to theinclination when the image of the affected part has been captured in thepast. The system control circuit 220 may display information regarding adifference between the inclination information regarding the imagingapparatus 200 in the image capturing for recording and the inclinationinformation regarding the imaging apparatus 200 in the live view. Theinformation regarding the difference may be generated by the systemcontrol circuit 220 of the imaging apparatus 200, or may be generated bythe information processing apparatus 300 and received by the imagingapparatus 200.

In step S511, the system control circuit 220 determines whether an imagecapturing instruction issued by the user pressing a shutter releasebutton included in the operation unit 224 is received.

If the image capturing instruction is received (YES in step S511), instep S512 and the subsequent steps, the processing proceeds to theprocess of capturing the image of the affected part for recording. Onthe other hand, if the image capturing instruction is not received (NOin step S511), the processing returns to step S503, and the processes ofstep S503 and the subsequent steps are performed. Thus, the processes ofsteps S503 to S511 are repeated until the image capturing instruction isreceived, whereby the imaging apparatus 200 continuously transmits theimage data in the live view to the information processing apparatus 300.Every time the imaging apparatus 200 transmits the image data, theimaging apparatus 200 receives, from the information processingapparatus 300, the image data obtained by superimposing the informationindicating the result of extraction of the affected region and theinformation regarding the size of the affected region on the image dataof the affected part.

In step S512, the AF control circuit 218 performs an AF process forcontrolling the driving of the lens group 212 so that the object comesinto focus. This process is similar to the process of step S503.

In step S513, in response to an image capturing instruction from theuser, the image capturing unit 211 captures an image of the object.Specifically, the image capturing unit 211 captures a still image of theaffected part for recording.

If it is determined in step S537 that the object information 900corresponding to the target patient ID is not stored, the system controlcircuit 220 may prompt the user to first capture an image of theaffected part for recording and then capture an image of the posture ofthe object. Specifically, the system control circuit 220 adjusts themagnification of the image capturing unit 211 so that, after theaffected part is captured, the entire body of the object is captured.Then, the user performs image capturing. In a case where the posture ofthe object is thus automatically captured, the process of capturing theposture of the object in step S502 can be omitted. Informationindicating that the object information 900 corresponding to the targetpatient ID is not stored can be received from the information processingapparatus 300 in step S509.

In step S514, the image processing circuit 217 acquires data of thecaptured image and performs a development process and a compressionprocess on the image data, thereby generating image data based on theJPEG standard, for example. This process is similar to the process ofstep S505. However, to give priority to accuracy of the measurement ofthe affected region, it is desirable that the image processing circuit217 perform a resizing process for resizing the image data to a largersize than or the same size as the size of the image data in step S505.For example, the size of the image data subjected to the resizingprocess is approximately 4.45 megabytes in the case of 1440 pixels×1080pixels in 4-bit RGB colors. However, the size of the image data to besubjected to the resizing process is not limited to the above.

In step S515, the system control circuit 220 generates distanceinformation regarding the distance to the object. This process issimilar to the process of step S506.

In step S516, based on the inclination information output from theinclination detection device 225, the system control circuit 220generates inclination information regarding the imaging apparatus 200 inthe image capturing for recording. This process is similar to theprocess of step S507.

In step S517, the system control circuit 220 transmits the image data ofthe affected part subjected to the resizing process in step S514, thedistance information regarding the distance to the object that isgenerated in step S515, and the inclination information regarding theimaging apparatus 200 in the image capturing for recording that isgenerated in step S516, to the information processing apparatus 300 viathe communication device 219.

Then, the processing performed by the information processing apparatus300 will be described.

In step S541, the CPU 310 of the information processing apparatus 300receives, via the communication device 313, the image data of theaffected part, the distance information regarding the distance to theobject, and the inclination information regarding the imaging apparatus200 in the image capturing for recording that are transmitted from theimaging apparatus 200.

In step S542, using the auxiliary calculation device 317, the CPU 310extracts an affected region from the received image data of the affectedpart (segments the affected region and another region). This process issimilar to the process of step S532.

In step S543, the calculation device 311 of the CPU 310 calculates thearea of the affected region as information regarding the size of theextracted affected region. This process is similar to the process ofstep S533.

In step S544, the calculation device 311 calculates evaluationinformation regarding the affected region. Specifically, based on thelength on the focal plane corresponding to one pixel on the image thatis obtained in step S543, the calculation device 311 calculates thelengths of the major axis and the minor axis of the extracted affectedregion and the area of a rectangle circumscribing the affected region.DESIGN-R as an evaluation indicator for a pressure ulcer defines thatthe size of a pressure ulcer is to be obtained by measuring the value ofthe product of the major axis and the minor axis. The image processingsystem 1 according to the present exemplary embodiment analyzes themajor axis and the minor axis and thereby can secure compatibility withdata measured by DESIGN-R in the past. Since DESIGN-R does not provide astrict definition, a plurality of calculation methods is mathematicallypossible as the calculation method for calculating the major axis andthe minor axis.

As a first example of the calculation method for calculating the majoraxis and the minor axis, the calculation device 311 calculates arectangle having the smallest area (a minimum bounding rectangle) amongrectangles circumscribing the affected region. Next, the calculationdevice 311 calculates the lengths of the long side and the short side ofthe rectangle, and uses the length of the long side as the major axisand the length of the short side as the minor axis in calculation. Then,based on the length on the focal plane corresponding to one pixel on theimage that is obtained in step S543, the calculation device 311calculates the area of the rectangle.

As a second example of the calculation method for calculating the majoraxis and the minor axis, the calculation device 311 selects the maximumFeret's diameter that is the maximum caliper length as the major axis,and selects the minimum Feret's diameter as the minor axis.Alternatively, the calculation device 311 may select the maximum Feret'sdiameter that is the maximum caliper length as the major axis, andselect a length measured in a direction orthogonal to the axis of themaximum Feret's diameter as the minor axis.

As the calculation method for calculating the major axis and the minoraxis, any method can be selected based on compatibility withconventional measurement results.

The process of calculating the lengths of the major axis and the minoraxis of the affected region and the area of the rectangle is notexecuted on the image data received in step S531. The live view isintended to enable the user to confirm the result of extraction of theaffected region. Thus, during the live view, an image analysis processcorresponding to step S544 on the image data received in step S531 isomitted, thereby reducing the processing time.

In step S545, the image processing circuit 217 generates image dataobtained by superimposing information indicating a result of extractionof the affected region and information regarding the size of theaffected region on the image data from which the affected region is tobe extracted. The information regarding the size of the affected regionin this step includes the evaluation information regarding the affectedregion, such as the major axis and the minor axis of the affectedregion.

FIGS. 8A, 8B, and 8C are diagrams illustrating the method forsuperimposing the information indicating the result of extraction of theaffected region and the information regarding the size of the affectedregion including the major axis and the minor axis of the affectedregion on the image data. Since a plurality of pieces of informationregarding the size of the affected region is expected, a description isgiven with reference to FIGS. 8A to 8C.

An image 801 illustrated in FIG. 8A is obtained using the minimumbounding rectangle as the calculation method for calculating the majoraxis and the minor axis. As with FIG. 7B, at the upper left corner ofthe image 801, as the information regarding the size of the affectedregion, the label 711 is superimposed in which the character string 712indicating the area of the affected region is displayed in whitecharacters on a black background.

At the upper right corner of the image 801, as the information regardingthe size of the affected region, a label 812 is superimposed in whichthe major axis and the minor axis calculated based on the minimumbounding rectangle are displayed.

The label 812 includes character strings 813 and 814. The characterstring 813 indicates the length of the major axis (in centimeters (cm)).The character string 814 indicates the length of the minor axis (incentimeters). In the image 801, a rectangular frame 815 representing theminimum bounding rectangle is superimposed on the affected region. Therectangular frame 815 is superimposed together with the lengths of themajor axis and the minor axis, whereby the user can confirm in whichportion in the image the lengths are measured.

At the lower right corner of the image 801, a scale bar 816 issuperimposed. The scale bar 816 is used to measure the size of theaffected part 102, and the size of the scale bar 816 relative to theimage data is changed based on the distance information. Specifically,the scale bar 816 is a bar graduated up to 5 cm at 1-cm intervals basedon the length on the focal plane corresponding to one pixel on the imagethat is obtained in step S543, and corresponds to the size on the focalplane of the imaging apparatus 200, i.e., on the object 101. The userreferences the scale bar 816 and thereby can grasp the size of theobject 101 or the affected part 102.

At the lower left corner of the image 801, an indicator 817 for Sizeevaluation of DESIGN-R is superimposed. In the indicator 817 for Sizeevaluation of DESIGN-R, based on a numerical value obtained bymeasuring, in centimeters, the major axis and the minor axis (themaximum diameter orthogonal to the major axis) of the extent of skininjury and by multiplying the major axis and the minor axis, Size isclassified into the above-described seven levels. In the presentexemplary embodiment, the indicator 817 obtained by replacing the majoraxis and the minor axis with values output using respective calculationmethods for calculating the major axis and the minor axis issuperimposed.

An image 802 illustrated in FIG. 8B is obtained using the maximumFeret's diameter as the major axis and the minimum Feret's diameter asthe minor axis. At the upper right corner of the image 802, a label 822is superimposed in which a character string 823 indicating the length ofthe major axis and a character string 824 indicating the length of theminor axis are displayed. In the affected region of the image 802, anadditional line 825 corresponding to the measurement position of themaximum Feret's diameter and an additional line 826 corresponding to theminimum Feret's diameter are displayed. The additional lines 825 and 826as well as the character strings 823 and 824 indicating the lengths ofthe major axis and the minor axis are superimposed, whereby the user canconfirm in which portion in the image the lengths are measured.

In an image 803 illustrated in FIG. 8C, the major axis is the same asthat in the image 802, but the minor axis is not the minimum Feret'sdiameter and is a length measured in a direction orthogonal to the axisof the maximum Feret's diameter. At the upper right corner of the image803, a label 832 is superimposed in which the character string 823indicating the length of the major axis and a character string 834indicating the length of the minor axis are displayed. In the affectedregion of the image 803, the additional line 825 corresponding to themeasurement position of the maximum Feret's diameter and an additionalline 836 corresponding to the length measured in the directionorthogonal to the axis of the maximum Feret's diameter are displayed.

The various pieces of information to be superimposed on the image dataillustrated in FIGS. 8A to 8C may be any one of the pieces ofinformation or a combination of a plurality of the pieces ofinformation. The user may be allowed to select information to bedisplayed. The images illustrated in FIGS. 7A, 7B, 8A, 8B, and 8C aremerely examples, and the display forms, the display positions, thesizes, the fonts, the font sizes, or the font colors of the pieces ofinformation regarding the sizes of the affected part 102 and theaffected region, or the positional relationships between the pieces ofinformation can be changed to meet various conditions.

In step S546, the CPU 310 of the information processing apparatus 300transmits the information indicating the result of extraction of theaffected region and the information regarding the size of the affectedregion to the imaging apparatus 200 via the communication device 313. Inthe present exemplary embodiment, the CPU 310 transmits the image dataobtained by superimposing the information indicating the result ofextraction of the affected region and the information regarding the sizeof the affected region on the image data of the affected part that isgenerated in step S545 to the imaging apparatus 200.

In step S547, the CPU 310 reads the patient ID from the image data ofthe barcode tag. If the patient ID is already read in step S535, theprocess of step S547 can be omitted.

In step S548, the CPU 310 checks the read patient ID against the patientID of the object registered in advance, thereby acquiring informationregarding the name of the object. If the information regarding the nameof the object is already acquired in step S536, the process of step S548can be omitted.

In step S549, the CPU 310 adds information to theimage-capturing-date-and-time field 909, the image-data-of-affected-partfield 910, the evaluation information field 911, and the inclinationinformation field 912 in the affected part information 908 of the objectinformation 900 corresponding to the target patient ID. In step S550,the CPU 310 stores the resulting information in the storage device 312.

Specifically, the CPU 310 stores information regarding the date and timeof the image capturing performed in step S513 in theimage-capturing-date-and-time field 909. The CPU 310 also stores theimage data of the affected part received in step S541 in theimage-data-of-affected-part field 910. The CPU 310 also stores theevaluation information calculated in step S544 in the evaluationinformation field 911. The CPU 310 also stores the inclinationinformation regarding the imaging apparatus 200 in the image capturingfor recording that is received in step S541 in the inclinationinformation field 912. As described with respect to the objectinformation 900 in FIG. 9A, based on the inclination information storedin the inclination information field 912, the CPU 310 can store orupdate the inclination information in the second inclination informationfield 907 in the posture information 903.

If the object information corresponding to the target patient ID is notstored in the storage device 312, the CPU 310 generates objectinformation corresponding to the patient ID and the informationregarding the name of the object, and stores information in the postureinformation 903 and the affected part information 908 of the objectinformation 900.

If the object information corresponding to the target patient ID isstored in the storage device 312, the CPU 310 may determine whether theimage data already stored in the image-data-of-posture field 905 and theimage data of the posture obtained in step S502 in the current imagecapturing match each other. These pieces of image data matching eachother means that the postures of the object included in both pieces ofimage data are the same. Thus, for example, if the object included inone of the pieces of image data takes a prone posture and the objectincluded in the other image data takes a recumbent posture, the CPU 310determines that the pieces of image data do not match. If the pieces ofimage data do not match, the CPU 310 updates the image data alreadystored in the image-data-of-posture field 905 with the image data of theposture obtained in step S502 in the current image capturing, and storesthe updated image data. The CPU 310 may update and store not only theimage data of the posture, but also at least either of the posture iconfield 904 and the first inclination information field 906 in the postureinformation 903.

Then, the processing performed by the imaging apparatus 200 will bedescribed.

In step S518, the system control circuit 220 of the imaging apparatus200 receives, via the communication device 219, the image data obtainedby superimposing the information indicating the result of extraction ofthe affected region and the information regarding the size of theaffected region on the image data of the affected part that istransmitted from the information processing apparatus 300.

In step S519, the system control circuit 220 displays, on the displaydevice 223, the image data obtained by superimposing the informationindicating the result of extraction of the affected region and theinformation regarding the size of the affected region on the receivedimage data of the affected part, for a predetermined time. In this step,the system control circuit 220 displays any of the images 801 to 803illustrated in FIGS. 8A to 8C, and if the predetermined time elapses,the processing returns to step S503.

As described above, in the present exemplary embodiment, in a case wherethe user captures an image of an affected part using the imagingapparatus 200, the user is notified of posture information regarding anobject obtained when an image of the affected part of the same objecthas been captured in the past, whereby the user can capture the image ofthe image of the affected part by setting the posture of the object tothe same posture as that when the object has been captured in the past.Thus, the user can capture an image with which the user can compareprogress more accurately.

In the present exemplary embodiment, DESIGN-R (registered trademark) isused as an evaluation indicator of a pressure ulcer. However, thepresent invention is not limited to this. Alternatively, anotherevaluation indicator such as the Bates-Jensen Wound Assessment Tool(BWAT), the Pressure Ulcer Scale for Healing (PUSH), or the PressureSore Status Tool (PSST) may also be used.

(First Modification)

In step S502 in the flowchart in FIG. 5A, a case has been describedwhere the image of the posture of the object is captured. However, thepresent invention is not limited to this case. For example, aconfiguration may be employed in which, in step S502, the imagingapparatus 200 allows the user to select the posture of the object.Specifically, in step S502, the system control circuit 220 displays theposture icons 921 to 924 illustrated in FIG. 9B or character informationindicating postures in a selectable manner on the display device 223.Thus, the user can select a posture icon or character informationcorresponding to the posture of the object. In step S508, the systemcontrol circuit 220 transmits the posture icon (including identificationinformation regarding the posture icon) or the character informationselected by the user to the information processing apparatus 300.

The user is thus allowed to select the posture of the object, whereby itis possible to easily identify the posture of the object. Further, theprocess of transmitting and receiving the image data of the posture canbe omitted. Thus, it is possible to reduce a processing load on theimage processing system 1.

(Second Modification)

In step S538 in the flowchart in FIG. 5A, a case has been describedwhere the posture information 903 of the object information 900 istransmitted to the imaging apparatus 200 to notify the user of theposture of the object taken when the image of the affected part has beencaptured in the past. However, the present invention is not limited tothis case. For example, if it is determined in step S537 that the objectinformation 900 corresponding to the target patient ID is not stored inthe storage device 312, the CPU 310 need not transmit the postureinformation 903 to the imaging apparatus 200. This is because, in thecase where the object information 900 corresponding to the targetpatient ID is not stored in the storage device 312 in step S537, theobject is captured for the first time this time, and therefore, it isless necessary to notify the user of the posture of the object takenwhen the image of the affected part has been captured in the past.

(Third Modification)

In step S510 in the flowchart in FIG. 5, a case has been described wherethe system control circuit 220 displays, on the display device 223, theposture of the object taken when the image of the affected part has beencaptured in the past. However, the present invention is not limited tothis case. For example, the system control circuit 220 may notify theuser of the posture of the object taken when the image of the affectedpart has been captured in the past by sound using a sound device (notillustrated).

While the present invention has been described above together withvarious exemplary embodiments and modifications, the present inventionis not limited to the above exemplary embodiments and modifications andcan be changed within the scope of the present invention. The aboveexemplary embodiments and modifications may be combined together whenappropriate. For example, a target analyzed by the informationprocessing apparatus 300 is not limited to an affected part, and may bean object included in image data.

The present invention is not limited to the above exemplary embodiments,and can be changed and modified in various manners without departingfrom the spirit and the scope of the present invention. Thus, thefollowing claims are appended to publicize the scope of the presentinvention.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

1. An imaging apparatus comprising: an image capturing unit; and acontrol unit configured to perform control to, in a case where postureinformation regarding an object obtained when an image of an affectedpart of the object has been captured in the past is acquired and theimage capturing unit captures an image of the affected part of theobject, notify a user of the posture information regarding the objectand inclination information regarding the imaging apparatus.
 2. Theimaging apparatus according to claim 1, wherein the control unitdisplays the posture information regarding the object and theinclination information regarding the imaging apparatus on a displaydevice.
 3. The imaging apparatus according to claim 2, wherein thecontrol unit displays at least any one of a display item schematicallyillustrating a posture of the object, image data obtained by capturingan image of the posture of the object, inclination information regardingthe imaging apparatus obtained when the image of the posture of theobject is captured, and character information representing the postureof the object in a character, as the posture information on the displaydevice.
 4. The imaging apparatus according to claim 2, wherein theaffected part is a pressure ulcer, and wherein the posture informationincludes information that allows identification of at least any one of aprone posture, a recumbent posture, and a sitting posture of the object.5. The imaging apparatus according to claim 2, wherein the control unitdisplays, on the display device, the posture information regarding theobject and the inclination information regarding the imaging apparatusin a superimposed manner on image data in live view captured by theimage capturing unit.
 6. The imaging apparatus according to claim 2,wherein the control unit transitions a screen on which image data inlive view captured by the image capturing unit is displayed to adifferent screen in response to a user operation, thereby displaying theposture information regarding the object and the inclination informationregarding the imaging apparatus on the display device.
 7. The imagingapparatus according to claim 5, wherein the image data in the live viewcaptured by the image capturing unit is image data transmitted from theimaging apparatus to an external apparatus, subjected to imageprocessing by the external apparatus, and then received from theexternal apparatus.
 8. The imaging apparatus according to claim 1,further comprising a communication unit configured to transmitidentification information regarding the object to an externalapparatus, wherein the control unit controls the communication unit totransmit the identification information to the external apparatus andreceive the posture information and the inclination informationassociated with the identification information from the externalapparatus.
 9. The imaging apparatus according to claim 1, furthercomprising a communication unit configured to, in a case where the imagecapturing unit captures the image of the affected part of the object,transmit the posture information regarding the object and theinclination information regarding the imaging apparatus to an externalapparatus, wherein the control unit receives, from the externalapparatus via the communication unit, the posture information and theinclination information transmitted from the communication unit andstored in the external apparatus.
 10. The imaging apparatus according toclaim 9, wherein the communication unit transmits image data obtained bythe image capturing unit capturing an image of a posture of the objectto the external apparatus.
 11. The imaging apparatus according to claim9, wherein the communication unit transmits posture information selectedfrom a plurality of pieces of posture information by the user to theexternal apparatus.
 12. An information processing apparatus comprising:a communication unit configured to receive identification informationregarding an object from an imaging apparatus and transmit, to theimaging apparatus, posture information regarding the object obtainedwhen an image of an affected part of the object has been captured in thepast and inclination information regarding the imaging apparatus thatare associated with the identification information.
 13. The informationprocessing apparatus according to claim 12, wherein the identificationinformation regarding the object and the posture information regardingthe object are stored in association with each other in a storagedevice, wherein the communication unit receives the identificationinformation regarding the object and the posture information regardingthe object from the imaging apparatus, and wherein, in a case where theposture information stored in the storage device in association with thesame identification information as the identification informationreceived by the communication unit does not match the postureinformation received by the communication unit, the control unit updatesthe posture information stored in the storage device with the postureinformation received by the communication unit.
 14. The informationprocessing apparatus according to claim 12, wherein the communicationunit receives the identification information regarding the object andposture information regarding the object from the imaging apparatus, andwherein in a case where object information corresponding to theidentification information received by the communication unit is notstored in a storage device, the control unit stores the identificationinformation and the posture information received by the communicationunit in association with each other in the storage device.
 15. An imageprocessing system comprising: an information processing apparatusincluding: a communication unit configured to receive identificationinformation regarding an object from an imaging apparatus and transmit,to the imaging apparatus, posture information regarding the objectobtained when an image of an affected part of the object has beencaptured in the past and inclination information regarding the imagingapparatus that are associated with the identification information; andthe imaging apparatus including: an image capturing unit; and a controlunit configured to, in a case where the image capturing unit captures animage of the affected part of the object, notify a user of the postureinformation regarding the object and the inclination informationregarding the imaging apparatus that are transmitted from thecommunication unit.
 16. A control method for controlling an imagingapparatus, the control method comprising: acquiring posture informationregarding an object obtained when an image of an affected part of theobject has been captured in the past, and inclination informationregarding the imaging apparatus; performing control to notify a user ofthe posture information regarding the object and the inclinationinformation regarding the imaging apparatus; and capturing an image ofthe affected part of the object.
 17. A control method for controlling aninformation processing apparatus, the control method comprising:receiving identification information regarding an object from an imagingapparatus; and transmitting, to the imaging apparatus, postureinformation regarding the object obtained when an image of an affectedpart of the object has been captured in the past and inclinationinformation regarding the imaging apparatus that are associated with theidentification information.
 18. A control method for controlling animage processing system including an imaging apparatus and aninformation processing apparatus, the control method comprising:transmitting identification information regarding an object from theimaging apparatus to the information processing apparatus; transmitting,from the information processing apparatus to the imaging apparatus,posture information regarding the object obtained when an image of anaffected part of the object has been captured in the past andinclination information regarding the imaging apparatus that areassociated with the identification information; performing control tonotify a user of the posture information regarding the object and theinclination information regarding the imaging apparatus that areacquired by the imaging apparatus; and capturing an image of theaffected part of the object.